1. Field of the Invention
The present invention relates to an imaging device and an imaging method for forming subject images on an image sensor, which have respectively passed through different areas of a photographic lens in two directions, to acquire different viewpoint images. The present invention is more particularly relates to an imaging device and an imaging method for stereoscopically displaying a photographing angle of view of an image having parallax as a through image.
2. Description of the Related Art
Conventionally, there has been a stereoscopic imaging device configured to form subject images, which have respectively passed through different areas of a photographic lens in two directions, on an image sensor to acquire different viewpoint images.
An optical system shown in FIG. 7 is configured to cause a mirror 4 to pupil-divide subject images that have passed through different horizontal areas of a main lens 1 and a relay lens 2, and to form the respective subject images on image sensors 7 and 8 through image formation lenses 5 and 6.
(A) to (C) portions of FIG. 8 are views showing the state of separation of images formed on the image sensor in the case of front focus, in-focus (best focus), and rear focus states. Note that in FIG. 8, the mirror 4 shown in FIG. 7 is omitted in order to compare differences in image separation depending on the focus states.
Among the pupil-divided images, the in-focus images are formed at the same position (are aligned) on the image sensor as shown in (B) portion of FIG. 8, whereas the front-focus and rear-focus images are formed at different positions (are separated) on the image sensor as shown in (A) and (C) portions of FIG. 8.
Therefore, by acquiring subject images which have been pupil-divided in the horizontal direction through the image sensors 7 and 8, it becomes possible to acquire a left viewpoint image and aright viewpoint image (3D image) having different viewpoints depending on subject distances.
According to Japanese Patent Application Laid-Open No. 2007-104248, an electronic camera has an imaging unit, a light amount distribution detection unit, an image shift amount detection unit, and an image processing unit. The imaging unit is configured to photoelectrically convert a subject image obtained by the photographing optical system to generate photographed image data. The light amount distribution detection unit detects respective light amount distributions of luminous fluxes which have come from the subject and have passed through different optical paths. The image shift amount detection unit detects an image shift amount in a plurality of points within a photographing screen based on the light amount distributions. The image processing unit is configured to change, in the horizontal direction, positional relationship of the subject in the photographed image data based on the image shift amount in the plurality of points to generate stereogram image data. The image processing unit is also configured to change, in the horizontal direction, the positional relationship of the subject in the stereogram image data based on an input from an input unit to correct the stereogram image data. Since the stereogram image data can be corrected, it becomes possible to acquire a natural stereogram image, which causes less discomfort for users.
Japanese Patent Application Laid-Open No. 2009-124308 discloses a method for generating a multi-viewpoint image, including: performing pattern matching between left-eye and right-eye stereo images photographed with two cameras on the left side and right side, using one of the stereo images as reference; specifying a matched image for each pixel; calculating an intermediate position image by interpolating an interpixel distance between images matched for each of the left and right pixels; and calculating a right outer image and a left outer image by extrapolating the interpixel distance between the images matched for each of the left and right pixels.
In Japanese Patent Application Laid-Open No. 2008-92007, paragraphs 0047 to 0048, and 0071, a stereo matching processing unit (object detection means) 13 detects one or more corresponding points (objects) corresponding to each other in search spaces set by the aforementioned space setting unit 42 in two pieces of image data A and B which have been each processed by image processing units 4A and 4B and stored in a memory section 31. A distance calculation unit (position calculation means) 44 calculates three-dimensional coordinate values (position information) of the corresponding points detected by the above-stated stereo matching processing unit 83. Photographic lenses 21A2 and 21B2 in photographing units 20A and 20B are different in configuration. The photographic lens 21A2 includes a zoom lens and an unshown zoom lens driving unit (driving means) that drives the zoom lens, while the photographic lens 21B2 includes a fixed-focus lens with an angle of view equal to a wide angle end of the zoom lens of the photographic lens 21A2. Such configuration is employed to achieve cost reduction.
Japanese Patent Application Laid-Open No. 2003-7994, Japanese Patent Application Laid-Open No. 2001-12916, and Japanese Patent Application Laid-Open No. 2001-016611 are examples of a three-dimensional image generation technology with use of a single optical system. For example, Japanese Patent Application Laid-Open No. 2003-7994 discloses a solid-state image sensor configured to include a large number of pixels arrayed on the same imaging surface and to photoelectrically convert a subject image formed on the imaging surface so as to generate an image signal. In the solid-state image sensor, the large number of pixels is divided into two groups, and pixels in the respective groups are made to have different light incident angles.
Japanese Patent Application Laid-Open No. 08-331607, Japanese Patent Application Laid-Open No. 2008-141666, Japanese Patent Application Laid-Open No. 2009-14445, Japanese Patent Application Laid-Open No. 2008-116309, and Japanese Patent Application Laid-Open No. 2000-102040 disclose examples of a method for searching corresponding points between different viewpoint images, a technology for acquiring depth information by stereo matching, and a three-dimensional image generation technology involving use of two-dimensional images and distance information (depth information).
In Japanese Patent Application Laid-Open No. 2008-271241, the focus is placed only on a main subject and portions other than the main subject are intentionally defocused. Accordingly, a plurality of images are obtained by successively photographing images while discretely moving focal positions by a predetermined moving amount, and the thus obtained images are synthesized so that a blur-emphasized image can be obtained.
In Japanese Patent Application Laid-Open No. 2003-209727, an image to be photographed is divided into a plurality of areas, and the areas, which are to be a foreground and a background of a main subject, are imaged with a focal position shifted from a focal position corresponding to a distance of the areas. Images of respective areas are individually extracted from the images imaged for each area, and these images are synthesized to form one image. This makes it possible to easily photograph an image with a desired blur amount.
Japanese Patent Application Laid-Open No. 2010-226500 shows one example of a method for generating a parallax map.
In a pupil dividing-type stereoscopic imaging device configured to image parallax images by selectively acquiring image information that have passed through different pupil positions in image formation means, to generate a stereoscopic image as in Japanese Patent Application Laid-Open No. 2003-7994, Japanese Patent Application Laid-Open No. 2001-12916, and Japanese Patent Application Laid-Open No. 2001-016611, parallax becomes 0 at the in-focus position, while at the out-of-focus position, parallax is generated corresponding to blurring (defocus amount).
Accordingly, in order to obtain a plane image without blurring in the stereoscopic imaging device, it is necessary to photograph an image with a narrowed diaphragm aperture and a deeper focus depth. However, if photographing is carried out with a narrowed diaphragm aperture, it is not possible to blur the foreground and the background in contrast to the main subject. In the pupil dividing-type stereoscopic imaging device, since photographing is carried out with a narrowed diaphragm aperture in order to obtain a clear plane image, creative image formation is limited.
The present invention provides a pupil dividing-type stereoscopic imaging device that can acquire a plane image with a desired portion, such as a foreground and a background, being blurred in contrast to a main subject, and a method for imaging plane image in the pupil dividing-type stereoscopic imaging device.